Background steps execution engine by lokesh755 · Pull Request #4476 · actions/runner

lokesh755 · 2026-06-04T02:31:57Z

Summary

Adds the core background step execution engine to the runner, enabling concurrent step execution within a single job. Background steps (background: true) run asynchronously while the foreground step loop continues. Control-flow steps (wait, wait-all, cancel) coordinate synchronization and lifecycle management.

Changes

New files

BackgroundStepCoordinator.cs — Coordinates background step execution, waiting, cancellation, and deferred state flushing. Manages a concurrency semaphore (default max 10).
ControlFlowStepData.cs — Data class for wait/wait-all/cancel steps.
BackgroundStepsL0.cs — Unit tests for background step coordination, waiting, cancellation, and failure propagation.

Modified files

StepsRunner.cs — Integrates BackgroundStepCoordinator; background steps are queued without blocking the main loop; control-flow steps are dispatched directly; implicit wait-all runs before post-hooks as a safety net.
JobExtension.cs — Initializes control-flow steps from BackgroundStepControl pipeline type; maps logical step IDs to external IDs for UI reference; creates implicit wait-all for uncovered background steps.
ExecutionContext.cs — Adds background step metadata as optional parameters to CreateChild

Key design decisions

Runner owns concurrency — run-service remains agnostic to step execution order
Non-blocking semaphore — foreground steps execute immediately even when all background slots are occupied
Implicit wait-all — safety net before post-hooks waits for any uncovered background steps

Copilot

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

TingluoHuang · 2026-06-06T00:38:27Z

+        public void InitializeCoordinator(int maxConcurrent)
+        {
+            _backgroundSteps.Clear();
+            _waitedStepIds.Clear();


is _waitedStepIds track the completed steps or uncompleted steps?

TingluoHuang · 2026-06-06T00:40:29Z

+        private async Task<TaskResult> WaitForStepsAsync(string[] stepIds, CancellationToken cancellationToken)
+        {
+            var ids = stepIds ?? Array.Empty<string>();
+            await WaitForStepTasksAsync(ids, cancellationToken);
+            return CompleteWaitedSteps(ids);
+        }
+
+        private async Task<TaskResult> WaitForAllAsync(CancellationToken cancellationToken)
+        {
+            var remaining = _backgroundSteps.Keys.Where(id => !_waitedStepIds.Contains(id)).ToList();
+            await WaitForStepTasksAsync(remaining, cancellationToken);
+            return CompleteWaitedSteps(remaining);
+        }


do we still want to have 2 extra private methods? it feels like we can just inline the code to where it used?

TingluoHuang · 2026-06-06T00:41:11Z

+            Trace.Info($"Background step '{stepId}' queued (slot will be acquired asynchronously).");
+        }
+
+        private async Task ExecuteBackgroundStepCoreAsync(


we can probably move all private method under the Private helpers section down.

TingluoHuang · 2026-06-06T00:42:27Z

+
+        private async Task CancelStepsAsync(string[] cancelStepIds)
+        {
+            if (cancelStepIds == null || cancelStepIds.Length == 0) return;


Suggested change

if (cancelStepIds == null || cancelStepIds.Length == 0) return;

if (cancelStepIds == null || cancelStepIds.Length == 0)

{

return;

}

TingluoHuang · 2026-06-06T00:45:12Z

+
+            foreach (var id in cancelStepIds)
+            {
+                FlushDeferredState(id);


should we call CompleteWaitedSteps() instead?

TingluoHuang · 2026-06-06T00:55:04Z

+                if (step.ExecutionContext.Result == TaskResult.Failed)
+                {
+                    Trace.Info($"Propagating failure from background step '{step.ExecutionContext.ContextName}' to job result.");
+                    jobContext.Result = TaskResultUtil.MergeTaskResults(jobContext.Result, TaskResult.Failed);


can we defer the caller in steprunner to update jobcontext.result?

TingluoHuang · 2026-06-06T00:56:40Z

+        /// </summary>
+        public async Task RunControlFlowAsync(IExecutionContext stepContext, object data)
+        {
+            var controlFlow = data as ControlFlowStepData;


do we need to print anything to the output, so customer can know what we are doing under the step?

TingluoHuang · 2026-06-06T00:58:56Z

+                }
+                catch (TimeoutException)
+                {
+                    Trace.Info($"Some background steps did not terminate within {graceSeconds}s grace period.");


should we mark all steps that haven't finish as canceled?

TingluoHuang · 2026-06-06T00:59:52Z

+            {
+                FlushDeferredState(id);
+                _waitedStepIds.Add(id);
+                if (_backgroundSteps.TryGetValue(id, out var entry) && entry.Step.ExecutionContext.Result == TaskResult.Failed)


why we only care about TaskResult.Failed? what about canceled, etc.

TingluoHuang · 2026-06-06T01:02:58Z

+    /// Pure data for control-flow steps (wait, wait-all, cancel).
+    /// Type uses Pipelines.BackgroundControlTypes string constants.
+    /// </summary>
+    public sealed class ControlFlowStepData


should we rename this BackgroundStepControlFlowData?

lokesh755 force-pushed the lokesh755-background-steps-engine branch from 955429e to 13594a5 Compare June 4, 2026 03:11

lokesh755 force-pushed the lokesh755-thread-safety-primitives-background-stepps branch from becd2c1 to b88b481 Compare June 4, 2026 03:25

lokesh755 force-pushed the lokesh755-background-steps-engine branch 2 times, most recently from 4fbc66b to 95ab648 Compare June 4, 2026 15:26

lokesh755 marked this pull request as ready for review June 4, 2026 15:26

lokesh755 requested a review from a team as a code owner June 4, 2026 15:26

TingluoHuang reviewed Jun 4, 2026

View reviewed changes

Comment thread src/Runner.Worker/BackgroundStepCoordinator.cs Outdated

lokesh755 force-pushed the lokesh755-thread-safety-primitives-background-stepps branch from b88b481 to 5007995 Compare June 4, 2026 17:52

Base automatically changed from lokesh755-thread-safety-primitives-background-stepps to main June 4, 2026 18:08

Copilot AI review requested due to automatic review settings June 4, 2026 18:27

lokesh755 force-pushed the lokesh755-background-steps-engine branch from 95ab648 to 5a09827 Compare June 4, 2026 18:27

Copilot started reviewing on behalf of lokesh755 June 4, 2026 18:27 View session

Copilot AI reviewed Jun 4, 2026

View reviewed changes

lokesh755 force-pushed the lokesh755-background-steps-engine branch from 5a09827 to 265b98d Compare June 4, 2026 18:28

TingluoHuang reviewed Jun 4, 2026

View reviewed changes

Comment thread src/Runner.Worker/JobRunner.cs

TingluoHuang reviewed Jun 4, 2026

View reviewed changes

Comment thread src/Runner.Worker/JobExtension.cs Outdated

TingluoHuang reviewed Jun 4, 2026

View reviewed changes

Comment thread src/Runner.Worker/JobExtension.cs Outdated

lokesh755 force-pushed the lokesh755-background-steps-engine branch from 265b98d to c89cec5 Compare June 4, 2026 18:37

TingluoHuang reviewed Jun 4, 2026

View reviewed changes

Comment thread src/Runner.Worker/JobExtension.cs Outdated

TingluoHuang reviewed Jun 4, 2026

View reviewed changes

Comment thread src/Runner.Worker/JobExtension.cs Outdated

TingluoHuang reviewed Jun 4, 2026

View reviewed changes

Comment thread src/Runner.Worker/BackgroundStepContext.cs Outdated

TingluoHuang reviewed Jun 4, 2026

View reviewed changes

Comment thread src/Runner.Worker/BackgroundStepContext.cs Outdated

TingluoHuang reviewed Jun 4, 2026

View reviewed changes

Comment thread src/Runner.Worker/ControlFlowStepData.cs Outdated

TingluoHuang reviewed Jun 4, 2026

View reviewed changes

Comment thread src/Runner.Worker/FileCommandManager.cs

TingluoHuang reviewed Jun 4, 2026

View reviewed changes

Comment thread src/Runner.Worker/ExecutionContext.cs Outdated

lokesh755 changed the title ~~Background steps engine~~ Background steps execution engine Jun 4, 2026

TingluoHuang reviewed Jun 4, 2026

View reviewed changes

Comment thread src/Runner.Worker/ControlFlowStepData.cs

TingluoHuang reviewed Jun 4, 2026

View reviewed changes

Comment thread src/Runner.Worker/BackgroundStepCoordinator.cs Outdated

lokesh755 force-pushed the lokesh755-background-steps-engine branch 2 times, most recently from 50a2671 to 369bbe9 Compare June 4, 2026 22:14

lokesh755 force-pushed the lokesh755-background-steps-engine branch from 80ebb21 to b30b777 Compare June 5, 2026 01:11

TingluoHuang reviewed Jun 5, 2026

View reviewed changes

Comment thread src/Runner.Worker/StepsRunner.cs Outdated

TingluoHuang reviewed Jun 5, 2026

View reviewed changes

Comment thread src/Runner.Worker/StepsRunner.cs Outdated

lokesh755 force-pushed the lokesh755-background-steps-engine branch 2 times, most recently from 48753dd to 40c6e91 Compare June 5, 2026 10:01

TingluoHuang reviewed Jun 5, 2026

View reviewed changes

Comment thread src/Runner.Worker/ControlFlowStepData.cs Outdated

TingluoHuang reviewed Jun 5, 2026

View reviewed changes

Comment thread src/Runner.Worker/ControlFlowStepData.cs Outdated

TingluoHuang reviewed Jun 5, 2026

View reviewed changes

Comment thread src/Runner.Worker/ControlFlowStepData.cs Outdated

TingluoHuang reviewed Jun 5, 2026

View reviewed changes

Comment thread src/Runner.Worker/StepsRunner.cs Outdated

TingluoHuang reviewed Jun 5, 2026

View reviewed changes

Comment thread src/Runner.Worker/StepsRunner.cs Outdated

TingluoHuang reviewed Jun 5, 2026

View reviewed changes

Comment thread src/Runner.Worker/ExecutionContext.cs

lokesh755 force-pushed the lokesh755-background-steps-engine branch from 40c6e91 to 200d050 Compare June 5, 2026 22:44

TingluoHuang reviewed Jun 5, 2026

View reviewed changes

Comment thread src/Runner.Worker/BackgroundStepCoordinator.cs Outdated

TingluoHuang reviewed Jun 5, 2026

View reviewed changes

Comment thread src/Runner.Worker/BackgroundStepCoordinator.cs Outdated

lokesh755 force-pushed the lokesh755-background-steps-engine branch from d6aa1f6 to 975ab20 Compare June 5, 2026 23:15

Adds the core background step execution engine to the runner

c6e2142

lokesh755 force-pushed the lokesh755-background-steps-engine branch from 975ab20 to c6e2142 Compare June 5, 2026 23:32

lokesh755 added 3 commits June 5, 2026 20:03

Merge branch 'main' into lokesh755-background-steps-engine

a25a0dc

add test

e301339

cleanup

9a8ca18

TingluoHuang reviewed Jun 6, 2026

View reviewed changes

Conversation

lokesh755 commented Jun 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

New files

Modified files

Key design decisions

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

lokesh755 commented Jun 4, 2026 •

edited

Loading